Statistical Comparisons of Classifiers over Multiple Data Sets
نویسنده
چکیده
While methods for comparing two learning algorithms on a single data set have been scrutinized for quite some time already, the issue of statistical tests for comparisons of more algorithms on multiple data sets, which is even more essential to typical machine learning studies, has been all but ignored. This article reviews the current practice and then theoretically and empirically examines several suitable tests. Based on that, we recommend a set of simple, yet safe and robust non-parametric tests for statistical comparisons of classifiers: the Wilcoxon signed ranks test for comparison of two classifiers and the Friedman test with the corresponding post-hoc tests for comparison of more classifiers over multiple data sets. Results of the latter can also be neatly presented with the newly introduced CD (critical difference) diagrams.
منابع مشابه
On Combining Multiple Classifiers Using an Evidential Approach
Combining multiple classifiers via combining schemes or meta-learners has led to substantial improvements in many classification problems. One of the challenging tasks is to choose appropriate combining schemes and classifiers involved in an ensemble of classifiers. In this paper we propose a novel evidential approach to combining decisions given by multiple classifiers. We develop a novel evid...
متن کاملOn the use of Heronian means in a similarity classifier
This paper introduces new similarity classifiers using the Heronian mean, and the generalized Heronian mean operators. We examine the use of these operators at the aggregation step within the similarity classifier. The similarity classifier was earlier studied with other operators, in particular with an arithmetic mean, generalized mean, OWA operators, and many more. The two classifiers here ar...
متن کاملA study of statistical techniques and performance measures for genetics-based machine learning: accuracy and interpretability
The experimental analysis on the performance of a proposed method is a crucial and necessary task to carry out in a research. This paper is focused on the statistical analysis of the results in the field of genetics-based machine Learning. It presents a study involving a set of techniques which can be used for doing a rigorous comparison among algorithms, in terms of obtaining successful classi...
متن کاملComparative Performances of Stochastic Competitive Evolutionary Neural Tree (SCENT) with Neural Classifiers
A stochastic competitive evolutionary neural tree (SCENT) is described and evaluated against the best neural classifiers with equivalent functionality, using a collection of data sets chosen to provide a variety of clustering scenarios. SCENT is firstly shown to produce flat classifications at least as well as the other two neural classifiers used. Moreover its variability in performance over t...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید
ثبت ناماگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید
ورودعنوان ژورنال:
- Journal of Machine Learning Research
دوره 7 شماره
صفحات -
تاریخ انتشار 2006